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Abstract 

Background: Most work on the topic of activity landscapes has focused on their quantitative description and visual 
representation, with the ainn of aiding navigation of SAR. Recent developments have addressed applications such as 
quantifying the proportion of activity cliffs, investigating the predictive abilities of activity landscape methods and 
so on. However, all these publications have worked under the assumption that the activity landscape models are 
"real" (i.e., statistically significant). 

Results: The current study addresses for the first time, in a quantitative manner, the significance of a landscape or 
individual cliffs in the landscape. In particular, we question whether the activity landscape derived from observed 
(experimental) activity data is different from a randomly generated landscape. To address this we used the SALI 
measure with six different data sets tested against one or more molecular targets. We also assessed the significance 
of the landscapes for single and multiple representations. 

Conclusions: We find that non-random landscapes are data set and molecular representation dependent. For the 
data sets and representations used in this work, our results suggest that not all representations lead to non-random 
landscapes. This indicates that not all molecular representations should be used to a) interpret the SAR and 
b) combined to generate consensus models. Our results suggest that significance testing of activity landscape models 
and in particular, activity cliffs, is key, prior to the use of such models. 
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Background 

The notion of activity landscapes has been extensively 
discussed since it was first coined by Maggiora [1]. 
While such landscapes can be defined in many ways, 
they are represented fundamentally as two structural di- 
mensions that aim to capture as much as possible the 
information of the structures in chemical space, plus 
one activity dimension. This allows for intuitive visual 
depictions and in particular this view of landscapes is 
useful when discussing activity cliffs-discontinuities in 
the landscape that correspond to molecules that are very 
similar but differ significantly in activity. Most workers 
have devised alternative numerical characterizations of 
landscapes including Structure- Activity Similarity (SAS) 
and related maps [2,3], the Structure Activity Landscape 
Index (SALI) [4] and the Structure Activity Relationship 
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Index (SARI) [5]. On the basis of these definitions 
many applications have been described. For example, 
Guha and Van Drie described the use of SALI curves 
to characterize the ability of a predictive model to char- 
acterize the landscape and correctly predict activity cliffs 
[6,7] and the use of SALI to enable model selection [8]. 
Bajorath and co-workers have published a series of pa- 
pers that identify and characterize different types of 
SARs in datasets [9,10]. They also employ landscapes in 
the context of target and target family specificity [11,12], 
which serve to highlight the fact that activity cliffs are 
dependent on the target [13] This is further supported by 
the lack of SAR transfer among different target families 
[14]). Medina-Franco and co-workers have addressed 
the topic of consensus activity landscapes, which char- 
acterize landscapes using multiple structural representa- 
tions [15]. In general, it is clear that activity cliffs are 
very dependent on the context, which can be defined in 
multiple ways-by target, by chemical series or by molecu- 
lar representation. 
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A key aspect of the landscape paradigm is that molecu- 
lar representation can significantly alter the structure of 
the landscape. As pointed out by Shanmugasundaram 
et al [16] molecules that are similar in one representa- 
tion can be dissimilar in another. As a result, activity 
cliffs, which in various forms a primary focus of most ac- 
tivity landscape analyses, are generally representation 
dependent. Some approaches such as consensus ap- 
proaches [17] allow one to alleviate this problem by iden- 
tifying pairs of molecules that are activity cliffs in 
multiple representations (suggesting that they are "true" 
activity cliffs). Although one cannot eliminate activity 
cliffs that are formed by only one or few representations, 
for practical applications one can prioritize the SAR of 
the consensus activity cliffs [18]. 

In addition to a structural representation dependence 
of activity cliffs, there is also a dependence on activity. 
Past work has generally assumed that the observed activity 
together with the structure actually encodes a structure- 
activity relationship. For example, in traditional QSAR 
modeling, a key test of any model is its performance when 
faced with a permuted dependent variable (also known as 
y-scrambling) [19]. However, we are not aware of any work 
that has addressed the statistical significance of a landscape 
or individual cliffs in the landscape. While Guha et al. iden- 
tified significant cliffs, [8,20] this was simply based on a 
threshold (usually the 90th or 95th quantile of the ob- 
served SALT values) and did not actually address whether a 
cliff was a true cliff or an artificial cliff due to the molecular 
representation used or erroneous measures of activity [21]. 
In general, there has been little done to address whether 
landscapes, however they are defined, are significant in 
terms of being different from a random landscape. 

Validity and utility 

In absence of measures of significance, most approaches 
to activity landscapes have simply assumed that they are 
valid, and then have gone on to explore the utility of the 
landscape for various purposes. For example, it is as- 
sumed that reported activity data is correct and there 
are no attempts to assess if the associated landscapes are 
not different from random. In this context, a valid land- 
scape implies that an SAR is present. In this paper, we 
propose that methods that employ the landscape para- 
digm should first perform checks to assess the validity of 
the landscape. More specifically, is the landscape being 
employed different from one that could be generated by 
chance? Assuming that the landscape does pass such a 
test and is thus valid, one can then continue to assess 
the utility of the landscape for the task at hand. The 
concept of comparing predictive models to models de- 
veloped using random (or scrambled) data is common 
when developing and validating quantitative models such 
as QSAR equations [19]. However, using randomization 



methods have not been tested when investigating activity 
landscapes and their applications. 

In this study we also consider the validity of individual 
activity cliffs. Thus, having considered the significance of 
a landscape, we then go on to assess the significance of 
individual activity cliffs. While it is possible to envision 
several ways to address this issue (depending on the ac- 
tivity landscape method), in this work we specifically 
employ the SALI formulation of activity cliffs and com- 
bine the idea of threshold-based identification of SALI 
cliffs [4,8] with a permutation test to identify significant 
activity cliffs- significant in the sense of magnitude as 
well as statistics. To test the hypothesis whether a land- 
scape and its activity cliffs are valid, we used several data 
sets and different molecular representations previously 
used to generate activity landscapes. 

Results 

Significance testing for landscapes 

We evaluated the significance of entire landscapes for 
each endpoint/structure representation combination listed 
in Table 1. We first summarized individual datasets using 
the MACCS representations and a single target (when ac- 
tivity data for more than one target was available). Figure 1 
presents the results of the KS test, where the negative 
logio of the p-values are plotted so that taller bars indicate 
greater significance. The plot only shows the results ob- 
tained using the MACCS representation. It is evident that 
not all datasets lead to a landscape that is different from 
random (in a statistically significant sense). Similar results 
are observed if we consider other representations. Our 
data also indicate that there is no representation that con- 
sistently leads to non-random landscapes. There was no 
obvious correlation between representations (and thus 
chemical structure) and the significance of the landscape. 
One might expect that artifactual datasets (such as ones 
with artificially flat or jagged landscapes) would allow one 
to draw general conclusions and indeed such artificially 
generated datasets tend to be consistently insignificant. 
But in practice, it appears that such a priori conclusions 
cannot be made. 

We then considered landscapes for the same sets 
of molecules but against multiple target families, viz., 
the opioid receptor, carbonic anhydrase isoforms and 
monoamine transporters. In all cases, the same set of 
representations was employed and the results of the KS 
test are summarized in Figure 2. Once again it is clear 
that the results are dataset dependent. Interestingly, it 
appears that the radial descriptor representation tends 
to lead to non-significant landscapes, with only three 
datasets (CA I, CA IX and delta Opioid) exhibiting a 
non-random landscape with this descriptor. This observa- 
tion suggests that the landscape significance of a given 
dataset is not obviously correlated to the complexity of the 
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Table 1 Summary of datasets employed in this study 



Dataset 


Num. Obs. 


Targets 


Endpoints 


Representations 


Ref. 


BZD 


91 


Benzimidazole analogues tested against G. intestinollis (Gi) 


pICso 


MACCS, piDAPH3,TGD 


[22-25] 


BCG 


48 


Bicyclic guanidines with Kappa opioid receptor binding affinity 


pICso 


MACCS, piDAPHS, TGD 


[15] 


OP 


98 


Delta, Kappa, Mu opioid receptors 


pKi 


MACCS, Atompairs, Radial, piDAPHS 


[26,27] 


CA 


96 


CA 1, CA II, CA IX, CA XII 


pKi 


MACCS, Atompairs, Radial, piDAPHS 


[26,27] 



MT 299 Norepinephrine, serotonin and dopamine transporters Potency (nM) MACCS, Atompairs, Radial, piDAPHS [26,27] 



structure representation used in the calculation. To con- 
duct a comprehensive assessment of different fingerprints 
and their significance on activity landscapes, large scale 
studies will be required comparing several different finger- 
prints across diverse target families. 

In terms of cross -target analysis, Figure 2 suggests that 
the structural representation should be carefully se- 
lected, such that it leads to valid landscapes for all tar- 
gets that are to be compared. For example, in the case of 
the opioid receptor datasets, there is no representation 
that leads to a statistically significant landscape across 
all three targets. For the case of the CA datasets, it is 
only the atom pair and MACCS representations that 
lead to statistically non-random landscapes over all four 
targets. Of course, these conclusions apply for a given 
level of significance. Finally for the MT datasets, we see 
that the radial and piDAPHS representations lead to 
non-random landscapes, with p- values approaching zero, 
across all targets. 

Significance testing for individual SAL! values 

We next consider the significance of individual SALT 
values for the BCG and BZD datasets using MACCS, 
TGD and piDAPHS, summarized in Figure S. In each 
panel, the red points correspond to pairs of molecules 
for which the SALI value is calculated to be no different 
from random (at the 0.05 level). The vertical line is drawn 
at the 95th percentile of the SALI values for that dataset 
and representation. Note that from Table 2, all repre- 
sentations for these datasets do not lead to non-random 



landscapes at the 0.05 level of significance. However, even 
so, it is clear that SALI values for some pairs of compounds 
are deemed to be significant. At this point, we do not con- 
sider the per-SALI significance values for landscapes that 
are identified as non-significant. 

If we consider the piDAPHS representation for the 
BZD dataset it is clear that in the absence of the permu- 
tation test, all SALI values to the right of the vertical line 
would be considered "significant", simply in terms of 
magnitude. However, on the basis of the p-values and a 
pre-defined level of significance, only a subset of those 
points are different from random SALI values. For each 
SALI value that is identified as being significant (196 for 
the case of the BZD / piDAPHS combination), the 95% 
region of the random SALI population does not contain 
them. Thus an alternative way to define the test of sig- 
nificance is to state that a SALI value is significant if it is 
not contained between the 5th and 95th quantiles of the 
random SALI population (for that pair of molecules). 
Table 2 reports the fraction of SALI values that are identi- 
fied as significant in terms of magnitude only (Fmagnitude)> 
and the fraction that are significant in terms of magnitude 
as well as p-value (Fsignmcant)- Since we assume that non- 
significant landscapes should not be studied, we report 
Fsignificant = NA for those cases. 

Effects of combining representations 

The preceding analyses have focused on activity land- 
scapes generated from single structural representations. 
Given the fact that cliffs in one representation may not 
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Figure 1 A comparison of activity landscape significance for the datasets studied in this work, using MACCS fingerprints for structure 
representation. Tine dotted line represents p = 0.05. 
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Figure 2 A summary of the landscape significance test for three target families. The dotted line corresponds to p = 0.05 and so bars above 
this line correspond to landscapes that are different from random (in a statistically significant sense). 
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Table 2 A summary of KS test p-values for the landscape 
significance test for the BCG and BZD datasets using 
multiple representations 



Dataset 


Representation 


p-value 


^magnitude 


^significant 


BCG 


MACCS 


0.010 


0.063 


0.063 




piDAPHS 


0.307 


0.050 


NA 




TGD 


< 2.2E-16 


0.214 


0.211 


BZD 


MACCS 


0.098 


0.050 


NA 




piDAPHS 


0.063 


0.050 


NA 




TGD 


0..018 


0.050 


0.049 



Fmagnitude represent the fraction of SALI values that are identified as significant 
cliffs based on their magnitude being greater than a cutoff (95th percentile of 
the SALI values). Fsignificant is similar but only considers SALI values that are 
greater than the cutoff and exhibit a p-value < 0.05. If the landscape was not 
deemed significant, the latter was not reported. 



be cliffs in another, consensus approaches have been de- 
scribed [17], that employ multiple representations to 
evaluate the landscape. There are two approaches that 
one can tal<e in this scenario. First, one identifies com- 
pound pairs that are deemed activity cliffs in the land- 
scapes derived from the individual representations. Second, 
one can generate an aggregate representation by combin- 
ing the multiple individual representations [3]; In the first 
approach, one focuses the SAR analysis on pairs of com- 
pounds that are identified by most (or all if possible) repre- 
sentations. In the second case, one analyses the activity 
landscape derived from the aggregate representation. 

To investigate the effect of aggregate representations 
we consider the BCG and BZD datasets and various 



combinations of fingerprint representations. To com- 
bine the data we computed the mean similarity ("mean 
fusion") of different representations. This approach, 
based on the principles of data fusion, has been largely 
used to generate not only consensus models of the activ- 
ity landscape but also in many documented applications 
of data fusion [28-30]. Figure 4 summarizes the landscape 
significance test for the MACCS representation and three 
combination representations for the BCG and BZD. For 
the BCG dataset it appears that increasing the number of 
representations in the fusion does not necessarily lead to 
significant landscapes. For the BCG dataset, two repre- 
sentations (MACCS + TGD) lead to a non-random land- 
scape, but switching to MACCS + piDAPH3 does not. 
For both datasets, the three-representation case is identi- 
fied as non-random. Similar behavior is seen in the BZD 
dataset. Given that it is not clear as to how one might a 
priori judge whether a given combination of representa- 
tions will be significant (and the possible number of com- 
binations could easily become too large to examine 
explicitly), consensus landscapes (and cliffs) are probably 
best analyzed using multiple, individual representations 
and the landscapes derived from them. 

Discussion 

What are the implications of using random data in SAR 
studies using activity landscape approaches? What does 
it mean for a landscape to be random? One interpret- 
ation of such landscapes that emerge from the current 
approach is that they are "artifacts" and the structural 
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Figure 4 The results of the landscape significance test for the BCG and BZD datasets using a single representation and various 
combinations of representations. The horizontal line is drawn at p = 0.05. 
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representation does not reliably encode a real SAR; in 
other words, since similar landscapes can be generated 
from randomized activity and structural data there is no 
true connection between the structural features captured 
by the representation and the activity data. The analysis 
presented in this work has shown that for a given data 
set, there are structural representations that generate ac- 
tivity landscapes that are not different from random. It 
follows that conclusions obtained from non-significant 
landscapes are spurious. This interpretation is in line 
with the interpretation of QSAR models that fail the 
y-randomization test [19]. In contrast, non-random land- 
scapes suggest that there is a real association between the 
observed activity and the chemical space defined by the 
structural representation(s). 

Our analyses suggest that there is no clear a priori fea- 
ture of a dataset and its representation that would sug- 
gest that it is not different from random. Instead, one 
must perform a permutation test such as one described 
here to determine the validity or non-validity of the 
landscape. It should be noted that landscapes identified 
as non-random by the approach described here may not 
necessarily encode an SAR. An example of such a case 
would be a flat landscape, in which the activity of all 
molecules is (near) constant. While such a landscape is 
certainly not random, it does not represent a useful 
SAR. However, simply examining the activity and de- 
scriptor data readily identifies such anomalous landscapes. 
In such cases one would probably not consider them for 
further analyses. 

We have approached significance by comparing ob- 
served SALI distribution to random distribution of activ- 
ity data. Alternative would be to fit a distribution to the 
observed SALI values. But this is difficult since observed 
SALI distributions are rarely normal. One might con- 
sider the distributions of activity differences and dissimi- 
larity separately, but in general, automated distribution 
fitting will be difficult. Importantly, the random SALI 
distribution is derived from a scrambled dependent vari- 
able. We also considered a modification of the approach 
whereby we sampled the dependent variable in each 
randomization step from a normal distribution with the 
same mean and variance as the observed dependent 
variable. While the results were somewhat different from 
those reported here, we believe that this approach is not 
appropriate, as the observed dependent variable is usu- 
ally a mixture of a normal and long tailed distribution, 
due to the presence of activity cliffs. Thus sampling from 
a single normal distribution is unlikely to generate activ- 
ity cliffs during the simulation. 

While the aim of the proposed significance test is 
to justify the use of a landscape for further study, 
the test could also be used to prospectively identify suit- 
able descriptors for use in a landscape analysis. It is still 



debatable on what is "the best" structure representation 
to analyze activity landscapes. This issue is more pro- 
nounced when developing consensus models (e.g. com- 
bining representation with low linear correlation and/or 
representations with different design [3]). But in either 
case, it makes sense that at the very least the landscape 
should be different from random. The fact that one can 
perform the permutation test on single or combined 
representations allows one to rationally select represen- 
tations for landscape analysis. Indeed, earlier work has 
suggested the use of SALI as part of a model selection 
procedure^. While this ensures that the selected model 
captures the landscape, it is clear that significance test- 
ing should be incorporated to ensure that the model 
captures a valid landscape. In a similar vein, signifi- 
cance testing for individual SALI values (across different 
representations) is a systematic way to identify statis- 
tical significant activity cliffs that should be prioritized 
for SAR analysis/interpretation. Following the rationale 
of consensus models of activity landscapes, one can de- 
fine consensus significant activity cliffs as cliffs deemed 
significant by a number of different representations. 

Conclusions 

In this work we have presented an approach to address 
the important but currently overlooked question of 
whether the SAR captured in a model of activity landscape 
constructed with a given molecular representation(s) is 
different from random. Using the SALI formalism to 
quantif)^ the activity landscape and its activity cliffs, we 
have shown that some structure representations lead to 
activity landscapes that are indistinguishable from land- 
scapes generated using random activity data and ran- 
domized similarity values. For the current study that 
involved six different data sets with activity against one 
or more molecular targets, and different sets of 2D and 
3D fingerprints, we concluded that statistically significant 
landscapes are dataset and representation dependent (in 
addition to activity cliffs themselves being target and 
chemical series dependent). Testing the statistical signifi- 
cance of molecular representations is a rational approach 
to select molecular representations to generate robust ac- 
tivity landscape models and identify statistically signifi- 
cant activity cliffs. Of course, if observed activity data 
were available in replicate, one could more directly evalu- 
ate whether a landscape or cliff were different from noise. 
In absence of such data, we believe that the permutation 
tests described here is an appropriate test of the choice of 
representation employed in landscape activity analyses. 
While we have focused on the use of SALI, the proposed 
method is applicable to other metrics of the landscape 
that include an activity and similarity term in their for- 
mulation (such as SARI or SAS maps) and we hope that 
future studies involving activity landscapes will employ 
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this (or some other) form of significance testing before 
asking questions of such models. 

Methods 

Valid vs. non-valid landscapes 

The proposed approach is inspired by y-scrambling. 
Given a set of structures and their observed activity values, 
we evaluate the observed SALT matrix, defined by 



where Ai and Aj are the observed activities for molecules 
/ and and sim{ij) is the similarity between the two 
molecules (measured with a given molecular representa- 
tion and similarity metric). In contrast to y-scrambling 
for QSAR models, the dependent variable in the SALT 
formalism is a function of the activity and similarity. As 
a result, scrambling the dependent variable requires that 
one scramble both activity and similarity. In addition, as 
has been well documented, similarity measures depend 
critically on the molecular representations and similarity 
coefficients used [31,32]. Thus, in addition to the uncer- 
tainties in the activities, there are also uncertainties in 
the similarity values. But since similarities are not statis- 
tically independent, permuting the similarities directly is 
not possible. Instead we shuffle the fingerprints and then 
use the shuffled set to evaluate scrambled similarities. 
We then scrambled the activities and using the scram- 
bled similarity matrix recalculated the SALI matrix, re- 
peating this N times. Each such SALI matrix thus 
represents a "random" landscape. While scrambling the 
activity variable is a well accepted approach in permuta- 
tion tests (e.g., in QSAR models) there are number of 
ways one could consider randomization of the similarity 
values. The approach described here (i.e., shuffle the or- 
dering of the fingerprints, as opposed to scrambling the 
bits in the fingerprints themselves) is rather simple 
to implement. But more generally, one would replace 
scrambled similarities with a component that considered 
the significance of the similarity value. Given that there 
is no well accepted and generalized approach to the cal- 
culation of the significance of a chemical similarity, the 
development of such a method is beyond the scope of 
the current work and so we focus on activity and simi- 
larity scrambling as the primary driver of random land- 
scapes. The N SALI matrices are then converted to ID 
vectors with m{m -l)/2 elements where m is the num- 
ber of molecules in the dataset. The vectors are then 
concatenated, resulting in a single vector of length Nm 
(m - l)/2 elements. 

It is important to note that multiple scramblings are 
required to build up the distribution of random SALI 
values. Comparison of the observed SALI values to SALI 



values computed from a single randomization would not 
allow a robust differentiation between observed and ran- 
dom values (i.e., a single iteration could appear to be 
very similar to the observed SALI values, though highly 
unlikely). Note that if it were possible to analytically de- 
fine the distribution of SALI values, such an empirical 
process would not be required. However, based on 
examination of the current datasets the distribution sug- 
gests some form of a combination of normal and expo- 
nential distributions. It is unclear exactly how one might 
define such a mixture of distributions automatically for 
any dataset. 

The final step is to compare the distributions of the 
original (observed) SALI values and the ones calcu- 
lated from the N scrambled activities. We employ the 
Kolmogorov-Smirnov (KS) test [33] to perform this com- 
parison, in which the null hypothesis is that the two dis- 
tributions are the same. We select a such that if the 
p- value from the KS test is less than a we can reject the 
null hypothesis. In such a scenario we would claim that 
the observed landscape is different from random and that 
the difference is statistically significant. We term such a 
landscape as valid. 

We also investigated whether the individual random 
SALI matrices were distinguishable or indistinguishable 
from the accumulated distribution using the KS test. As 
expected, this was dataset dependent. Considering the 
MACCS fingerprints, the BCG dataset led to 1000 ran- 
dom SALI matrices being characterized as different from 
the accumulated distribution of SALI values. For the 
BZD dataset fingerprints, 103 out of 1000 random 
matrices were identified as different and for the MT 
(norepinephrine) dataset 727 were identified as different. 
Importantly, the random matrices identified as being dif- 
ferent from the accumulated distribution would not be 
considered valid, simply because they were constructed 
to be random. 

Significance of activity cliffs 

Assuming a landscape is valid, we can then consider 
individual pairs of molecules and assess whether they 
represent statistically significant activity cliffs. For this 
purpose we first defined a threshold SALI value such 
that pairs of molecules exhibiting a SALI greater than 
the threshold were deemed "significant" (in terms of 
magnitude) activity cliffs. For the selected pairs we then 
counted the number of times the random SALI value 
(described above) for that pair was greater than the ob- 
served value, denoted by Ngreater- The ratio Ngreaterl^ is 
taken as the empirical p-value and if less than a, the pair 
is deemed to be a statistically significant cliff. (Due to 
the scrambling procedure one can consider an arbitrary 
cell in the randomized SALI matrices or even all cells 
across all the randomized SALI matrices (in which case 
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the denominator would become Nm^) for the purposes 
of this calculation. For simplicity of calculation we con- 
sider randomized values for the pair in question). 

For the purposes of this study we selected the SALT 
threshold to be the 90th quantile of the observed SALT 
values for a given dataset. We also set A/' = 1000 and 
a = 0.05. All calculations were performed in R 2.15.0 
on a MacBook Pro (OS X 10.8.2, 2.7GHz Intel Core 17 
with 16GB of RAM). The multicore package was em- 
ployed to parallelize calculations. All code is available as 
Additional file 1. 

Datasets 

We considered six datasets, previously used to generate 
activity landscape models with sizes ranging from 47 to 
300 compounds. In some cases, the structures had been 
tested against multiple targets and thus multiple activity 
values were available. The structures were characterized 
by multiple fingerprint schemes frequently used in activ- 
ity landscape models including MACCS keys, radial fin- 
gerprints and pharmacophore fingerprints, available in 
the Molecular Operating Environment [15] and Canvas 
[34]. Table 1 summarizes the details of the datasets. 

MACCS keys are 166 bit binary fingerprints, where 
each bit position corresponds to a specific substructure. 
GpiDAPH3 is a 3-point pharmacophore based fingerprint 
computed from the two-dimensional graph. piDAPH4 is 
a 4-point pharmacophore based fingerprint computed 
from a three-dimensional conformation. Atompair fin- 
gerprints are differentiated by the type and distance sep- 
arating pairs of atoms. Radial fingerprints entail growing 
a set of fragments radially from each heavy atom over a 
series of iterations. Similarity matrices for observed activ- 
ities were taken from the previous published work sum- 
marized in Table 1. As discussed in these publications, the 
different fingerprint representations typically have different 
ranges of similarity values. 

Additional file 



Additional file 1: Code and data to reproduce figures. The ZIP file 
contains the R code and datasets required to perform the calculations 
and generate the figures for this article. It assumes you have R installed 
and are working on a Unix-like system. In addition you should have the 
ggplot2, grid, multicore and fingerprint R packages installed. To run the 
scripts execute run.sh from the command line. 
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